Peripheral component interconnect express (PCIe) is a connection system for connecting a host processor to PCIe devices in a PCIe system. In contrast to a peripheral component interconnect (PCI) system which uses a shared parallel bus for communication between PCI devices, a PCIe system uses point-to-point serial links for communication between PCIe devices. The PCIe links are organized in a tree structure topology that originates at a port of a root complex in the PCIe system and ends at PCIe endpoint devices (e.g., peripheral devices) in the PCIe system. To preserve compatibility with existing PCI standards, PCIe standards model physical point-to-point serial links as logical PCI busses. Thus in a typical PCIe system, PCIe devices are interconnected to each other through one or more logical PCI busses. In a PCIe system, the root complex, the PCIe devices, and the logical PCI busses are collectively referred to as a PCIe hierarchy. The portion of the PCIe hierarchy originating at the root complex port, which includes PCIe devices and logical PCI busses, is referred to as a bus hierarchy domain. Moreover, all PCIe devices in the bus hierarchy domain share a same address space of the bus hierarchy domain.
In some PCIe systems, multiple serial links are connected to each other through a PCIe switch. In these PCIe systems, the PCIe switch appears logically as a collection of PCI-to-PCI bridges interconnected by a virtual PCI bus. Moreover, each of the PCI-to-PCI bridges serves to logically connect two logical PCI busses. Because PCIe devices on both sides of a PCI-to-PCI bridge share the same address space of a bus hierarchy domain, such a PCI-to-PCI bridge is often referred to as a transparent PCI-to-PCI bridge. As a result, PCIe devices or address ranges on one side of the transparent PCI-to-PCI bridge cannot be isolated from access by PCIe devices on the opposite side of the transparent PCI-to-PCI bridge.
In a typical PCIe switch, a virtual PCI bus routes a packet between ports of the PCIe switch by broadcasting a packet received at a source connected to the virtual PCI bus to destinations connected to the virtual PCI bus. Each of the destinations connected to the virtual PCI bus has an address range in a bus hierarchy domain and monitors the virtual PCI bus for packets having destination addresses in that address range. If a destination address in a packet that is broadcast on the virtual PCI bus is in the address range of a destination connected to the virtual PCI bus, the destination accepts the packet. Otherwise, the destination ignores the packet. Although virtual PCI busses have been successfully employed for routing packets in PCIe switches, error recovery in a virtual PCI bus may be complex. For example, the virtual PCI bus must perform error recovery operations if none of the destinations connected to the virtual PCI bus accepts a packet broadcast on the virtual PCI bus.
Some types of PCIe systems include more than one bus hierarchy domain, each of which originates at a different root complex. Because the bus hierarchy domains are independent of each other, a PCIe device in one of the bus hierarchy domains is not visible to a PCIe device in another bus hierarchy domain. In these types of PCIe systems, a non-transparent PCI-to-PCI bridge is sometimes used to connect two logical PCI busses in the different bus hierarchy domains. Because each bus hierarchy domain has an independent address space and an independent device identifier space, the non-transparent PCI-to-PCI bridge performs address translation and device identifier translation between the two bus hierarchy domains connected to the non-transparent PCI-to-PCI bridge. Because the bus hierarchy domains separated by the non-transparent PCI-to-PCI bridge have independent address spaces, the PCIe system does not require a flat addressing model across the non-transparent PCI-to-PCI bridge.
Generally, a PCIe system routes a packet based on a destination address in the packet or a device identifier in the packet. In PCIe systems including a PCIe switch, the PCIe switch in the PCIe system routes a packet received from a source PCIe device in a bus hierarchy domain to a destination logical PCI bus in the same bus hierarchy domain based on a destination address or a device identifier in the packet. In PCIe systems including a non-transparent PCI-to-PCI bridge, the non-transparent PCI-to-PCI bridge forwards packets between a source bus hierarchy domain and a destination bus hierarchy domain. In this process, the PCI-to-PCI bridge receives a packet from a logical PCI bus in the source bus hierarchy domain, translates a destination address of the packet to a translated address in the destination bus hierarchy domain, generates a translated packet including the translated address, and forwards the translated packet to a logical PCI bus in the destination bus hierarchy domain. Although a non-transparent PCI-to-PCI bridge has been successfully employed in PCIe systems, such a non-transparent PCI-to-PCI bridge facilitates communication of packets between only two bus hierarchy domains.